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"Declining funds, declining test scores, declining standards, declining quality of 
products, declining resolve of legislators, state and local boards to fully fund programs 
without clear-cut objectives and documented outcomes" (Smith, 1984) have created a 
need for institutional assessment that will continue unabated into the 1990's. Thoughtful, 
well-planned and systematic reviews of both instructional and non-instructional 
programs provide community colleges with a way of determining whether programs are 
meeting stated objectives and what standards of performance should be maintained. 
Programs that need improvement or should be eliminated can be identified, and fiscal 
accountability can be achieved. 
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APPROACHES TO PROGRAM REVIEW 

Program reviews generally have qualitative and quantitative components. For the most 
part, the quantitative component utilizes the types of numerical data that are collected 
and reported to state agencies (e.g., student enrollment, weekly student contact hours, 
percentage of students completing the program, numbers of degrees granted, and 
numbers of students transferring to four-year institutions). Other categories for which 
data may be gathered include weekly student contact hours per full-time faculty 
equivalent, percentage of students obtaining jobs in their field of study, number of job 
openings in the service region, and full-time to part-time faculty ratio. 
In gathering qualitative data, students, faculty, advisory committee members, and other 
members of the college community who have knowledge of or experience with a 
program are asked to share their perceptions and judgments. Question may be 
open-ended, requiring the respondent to assess aspects of the program in his or her 
own words. However, most surveys depend heavily on ordinal or rank-order measures. 
Respondents are asked to rate aspects of a program's effectiveness as poor, below 
expectations, acceptable, good, or excellent-or some variation thereof. The evaluations 
typically focus on goals and objectives of the program, processes used in program 
implementation, and resources available for the program. 

Qualitative and quantitative components are integrated into an institution's overall plan 
for program review in various ways, ranging from heavily qualitative to heavily 
quantitative. 

THE QUALITATIVE APPROACH 

Voluntary self-studies conducted by internal review committees, augmented through 
validation studies by external review teams, lie at the qualitative end of the continuum. 
One example, Michigan's Program Review in Occupational Education (PROE), which 
was developed by a steering committee of local and state community college 
professionals and sponsored by the Michigan State Department of Education, includes: 
0 selection of a self-study coordinator and committee, and 

orientation meetings and announcements. 

0 completion of self-study instruments to determine faculty 

members' perceptions of program standards, strengths, 

and weaknesses; students' assessments of how well the 

program meets their needs; and advisory committee members' 

opinions of the occupational preparation of program 
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graduates. 

0 organization and presentation of tabulated and correlated 

responses into an understandable format. 

0 preparation of a written report sumnnarizing the program's 

strengths and weaknesses to assist decision making 

about the need to modify or redirect a program. 

0 preparation of an action plan for occupational program 

improvement. (Michigan Community Colleges, 1980). 

The colleges arrange for validation of the self-study by an outside team from other 
two-year colleges, industry, and the community. The team members, selected on the 
basis of their expertise and trained in applying the evaluation system, develop a 
consensus profile of the program, identifying where and why they agree or disagree with 
the self-study profile. 

These validation studies play an important role in lending credibility to voluntary, 
qualitative self-assessments, which are sometimes criticized as overly intuitive and 
self-serving (Rasor, 1983). In Michigan, concern for objectivity and credibility is 
addressed by the other components of the state's comprehensive evaluation system. 
These components-the Michigan Student Information System, Financial Information 
System, and Management Plan-provide quantitative data to supplement the qualitative 
findings. 

THE QUANTITATIVE APPROACH 

Maryland's program review process stands on the quantitative side of the 
qualitative/quantitative continuum. Under the direction of the Maryland State Board for 
Community Colleges, the colleges conduct annual reviews of both transfer and 
occupational programs. The Maryland program evaluation process begins with a 
quantitative review by the State Board of program data reported by the community 
colleges in the following categories: 
0 Name, status and code numbers of the program 

0 Enrollment and awards data for the last six years 

0 Enrollment and awards in similar programs at other colleges 

0 Graduate follow-up data 
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0 Annual job openings in the Baltimore area and statewide 
0 Discipline cost analysis data 

Each May, student, cost, and manpower information from the Program Data Monitoring 
(PDM) system is distributed to academic and occupational deans, and institutional 
research directors for verification. The State Board then identifies programs at each 
college that appear to be in difficulty and should be evaluated qualitatively. The 
identification process is assisted by "flags" in the PDM, indicating that a program has 
experienced declining enrollment or awards, low job placement or transfer rates, higher 
than average costs, or low student satisfaction levels. On the basis of these quantitative 
criteria checks, the Board develops question about the identified problems, which are 
forwarded to the college president for response. The questions are designed to 
determine why a program is in trouble and ways in which problems can be identified 
and addressed. The resulting program validation process may involve mail or telephone 
surveys of students, analyses of student transcripts, and assistance by the program 
advisory committee. The college must submit a written response to the State Board. On 
the basis of the program review, the Board may suggest the discontinuation or 
inactivation of a program. 

A study conducted at Pasadena City College (Carvell Education Management Planning, 
1982) represents a further step on the continuum toward quantification of the program 
review data. This program review model is based on six criteria: 

0 trends in weekly student contact hours (WSCH) 

0 faculty loads (WSCH/full-time equivalent faculty) 

0 class size 

0 student grade petitions 
0 student retention 

0 assigned cost of instruction per annual contact hour 

Programs are assessed in terms of how well they are performing currently in 
comparison with how well they performed in the past. Data for individual program 
performance are also compared with college-wide averages. The final aspect of 
program review at PCC is a qualitative assessment conducted by outside consultants 
based on personal interviews, a tour of the facilities, a review of records, and a written 
survey of faculty and administrators. 

Another model of quantitative program evaluation is provided by Foothill College in Los 
Altos, California. The two-part model was designed to examine instructional programs in 
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relation to program effectiveness, cost effectiveness and relation to the college mission 
(Lowe, 1983). Program and cost effectiveness are measured in terms of quantifiable, 
weighted criteria. For example, in assessments of program effectiveness, the number of 
students enrolled in relation to enrollment capacity is assigned a weight of 17 (the 
highest), while future employment outlook is given a weight of 6 (the lowest). In 
assessing cost effectiveness, average daily attendance has a weight of 25.5 (the 
highest), while the ratio of part-time to full-time instructors has a weight of 8 (the lowest). 

Through a formula based on percentile scores and the weights assigned each criteria 
for both program and cost effectiveness, a single score is obtained for each program 
evaluated. Programs are then ranked according to their overall scores. Finally, the 
programs are reviewed by the college president and the president's cabinet to judge 
their relation to the overall mission of the college. From this evaluation, decisions are 
made regarding the status of each program. During 1981-82, six of the ten lowest 
ranked programs were eliminated from the curriculum. 

Program reviews that rely heavily on quantitative analysis can be extremely threatening 
to staff members, particularly if declining enrollments or severe fiscal constraints 
mandate the elimination of programs and therefore reductions in teaching and support 
staff (Rasor, 1983). Both the Maryland State program review process and the Pasadena 
City College model provide recourse for those involved in programs that do not rank 
well on quantitative measures. These two systems address such issues as the need to 
provide general education courses for transfer students, the importance of responding 
to specific, but not necessarily quantifiable local community needs, and the need to 
provide a comprehensive curriculum through the qualitative component. 

The Foothill College model, which is heavily dependent upon quantitative data, provides 
almost no channel of response for the faculty and support staff in poorly rated 
programs, since the assessment of the programs' intangible and unquantifiable value 
rests solely with the president and members of the cabinet. 

Reviews of instructional programs require a balance between qualitative and 
quantitative components to ensure both accountability and fairness. 
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